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An approach based on the kernel methods for capturing the nonlinear interdependence 
between two signals is introduced. It is demonstrated that the proposed approach is useful 
for characterizing generalized synchronization with a successful simple example. An attempt 
to choose an optimal kernel parameter based on cross validation is also discussed. 

§1. Introduction 

Synchronization of chaotic systems has been explored extensively in recent years P 
In addition to complete synchronization between two identical chaotic systems^ var- 
ious notions of chaotic synchronization have evolvedP ' Among them, the concept of 
generalized synchronization (GS), which refers to a situation in which the states of 
two systems connected each other via a continuous mapping, has been introduced in 
order to study coherent behavior between two systems with different dynamics.^ 

Experimental detection of GS from data is a challenging problem. Because 
the synchronization manifold of GS has a highly nonlinear structure, conventional 
statistical tools such as the correlation coefficient does not work. 

Recently, the interest in the kernel methods has been stimulated in the machine 
learning community for analyzing data with nonlinearity in a unified manner.® Since 
the great success of Support Vector Machine, a considerable effort has been devoted 
to derive kernelization of various multivariate analysis methods. Therefore, it is 
meaningful to explore applicability of the kernel-based methods for analyzing non- 
linear dynamics. 

In this paper, we particularly employ Kernel Canonical Correlation Analysis 
(Kernel CCA p* for characterizing GS. We present an example for which Kernel CCA 
works successfully and also discuss how an optimal value of parameter of Kernel CCA 
can be chosen. 

§2. Kernel CCA 

Let us start with a formulation of Kernel CCA. For a pair of variables x € W 
and y £M q , Kernel CCA seeks a pair of nonlinear scalar functions / : M p — > K and 
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g : R 9 — > M such that the correlation coefficient 

cov(/(a),g(y)) ,„ n 

A? 7 = / — (2-1; 

v / var(/(x)) v / var( 5 (y)) 

between transformed variables is maximized. When a data set {(x n , y n )}n=i * s given, 
the maximal value of is estimated from the following procedure. 

Suppose that the nonlinear functions / and g are well approximated by linear 
combinations of kernels on data points (xi,yi) as f(x) = YliLi o>ik(xi,x) and g{y) = 
Y2i=i Pik{yi,y\- For example, a Gaussian kernel k(x,x') = exp(— \\x — x'\\ 2 /2a 2 ) is 
used as kernel!^* By substituting the above expressions for / and g into Eq. (|2T[) and 
replacing covariance cov(-, •) and variance var(-) with the empirical averages over the 
data set {{x n ,y n )}^ =1 , the maximization problem of Eq. (|2-1|) leads to the following 
generalized eigenvalue problemP 

K X K Y 
K Y K X 

where Kx,Ky are the Gram matrices (Kx)ij = k(xi,Xj) and (Ky)ij = k(yi,yj) 
determined from the given data set, and the term kI is introduced in order to avoid 
over-fitting. The first eigenvalue of Eq. (|2-2j) gives the maximal value p™ ax of pjr 
in Eq. (|2-ll) . p™ ax is called the canonical correlation coefficient, and the variables 
u = f(x), v = g(y) transformed by / and g are called the canonical variates of Kernel 
CCA. 

When the averages of {f(x n )}^ =1 and {g{x n )}^ =1 do not equal to zero, the 
Gram matrix K should be replaced with the following one: K = K — (l/iV)(j j)K — 
(l/N)K(j *j) + (l/iV 2 )(j t j)K(j *j), where j = *(1, 1, 1)P 
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§3. Results 



As an illustration, let us consider the following one-dimensional linear map driven 
by the two-dimensional baker's map: 



[xi(t + l),x 2 (t + l)] = < 



[ax 1 (t),x 2 (t)/b] 

if x 2 {t) < b, 
[a + {l-a) Xl (t),{x 2 {t)-b)/(l-b)} 
if x 2 (t) > b, 

y(t + l)=71/(t) + cos(27rxi(t)). 



(3-1) 
(3-2) 



The parameter of the baker's map Eq. ()3-lj) are taken as a = 0.3, b = 0.5 and 7 
in Eq. ()3-2|) is varied as the control parameter. For I7I < 1, the response system 
Eq. (|3-2|) is asymptotically stable for all (xi,x 2 ) in the unit square < x\,x 2 < 1, 
i.e., the system is in a state of GS. Since the natural measure of the baker's map is 
uniform in the x 2 direction, the driver-response relation in the system of Eqs. ()3T|) 
and (|3-2|) is visualized in the (x±,y) plane as shown in Fig. ^ We observe the 
transition from smooth to very complicated curves with the increase of 7P 
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Fig. 1. Projections of the strange attractors 
onto the (x\,y) plane. 7 = 0.2 (a), 0.4 
(b), 0.7 (c), 0.9 (d). 
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Fig. 2. Scatter plots of the first canonical vari- 
ates of Kernel CCA. 7 = 0.2 (a), 0.4 (b), 
0.7 (c), 0.9 (d). 



We apply Kernel CCA to the system of Eqs. (|3-1|) and (|3-2|) . Here, we employ 
a Gaussian kernel with a = 0.1 and k is set to 0.1. We prepare an orbit with length 
Af = 2x 10 3 as training data. Figure [21 shows scatter plots of the canonical variates 
(u n ,v n ) of Kernel CCA for four different values of 7 associated with Fig. ^ in 
Fig. |2 GS is clearly identified as a cloud of points along the diagonal on the plane 
of the canonical variates. By comparing graphs in Fig. [2] with those in Fig. the 
smaller value of the correlation coefficient between canonical variates corresponds to 
the complexity of the structure of the synchronization manifold. Figure |3] shows the 
canonical correlation coefficient pjr and the Lyapunov dimension Dl of the system 
of Eqs. (|3-1[) and (|3-2|) as functions of the control parameter 7. There is a monotonic 
relationship between p™ 3 * and Dl, which allows us to conclude that /9™ ax is a good 
index for characterization of GS. 

Next, we mention how the parameters of Kernel CCA can be chosen from data. 
In order to represent complicated nonlinear structures of the synchronization man- 
ifold via linear combinations of Gaussian kernels, the value of a should be chosen 
adequately. 

A naive way of choosing a is to maximize p™ ax estimated by the proposed 
method. The estimator of /3™ ax as a function of a is shown with the solid line in 
Fig. The index p™ ax increases monotonically with the decrease of cr, and p™ ax ~ 1 
is attained in the limit of a — > 0. Thus the maximization of p™ ax evaluated from 
the training data leads to the choice of the smallest value of cr, which results in 
over-fitting to the training data and spurious detection of GS. An adequate value of 
a is not determined in this way. 

A way to overcome this difficulty is to prepare another set of data ("testing 
data") separately from the training data, and evaluate p™ ax from the empirical aver- 
age over the testing data, while / and g are estimated from the training data. This 
strategy for testing the performance of the estimated model with new data is called 
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Fig. 3. The canonical correlation coefficient 
p™ ax and the Lyapnov dimension Dl as 
functions of 7. TV = 2 x 10 3 , a = 0.1 and 

K = 0.1. 
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Fig. 4. p™ ax as functions of a for 7 = 0.6. Or- 
bits with length N = 2x 10 2 and TV' = 10 4 
are used as training and testing data, re- 
spectively, k = 0.01. 



cross validation (CV)™ The dotted line in Fig. |I] shows the result of CV. The value 
of p™ ax shown with the dotted line takes its maximum at a value of a ^ 0. We 
expect that this value of a gives an optimal description of the system behind data. 

§4. Conclusions 

In summary, we have proposed a new approach for analyzing GS based on Kernel 
CCA with a successful application to a simple example. It is interesting to apply 
other kernel-based methods for analyzing various complex phenomena arising from 
nonlinear dynamical systems. 
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